Display system with image conversion mechanism and method of operation thereof

ABSTRACT

A method of operation of a display system includes: calculating a focus measure for an original image; calculating a segment mean based on the focus measure for a segment; generating an ordered segment based on the segment mean; generating a segment depth based on the ordered segment; and generating a three-dimensional image with the segment depth for displaying on a device.

TECHNICAL FIELD

The present invention relates generally to a display system and moreparticularly to a system for image conversion.

BACKGROUND ART

Modern consumer and industrial electronics, especially devices such asgraphical display systems, televisions, projectors, cellular phones,portable digital assistants, and combination devices, are providingincreasing levels of functionality to support modern life includingthree-dimensional display services. Research and development in theexisting technologies can take a myriad of different directions.

As users become more empowered with the growth of three-dimensionaldisplay devices, new and old paradigms begin to take advantage of thisnew device space. There are many technological solutions to takeadvantage of this new display device opportunity. One existing approachis to display three-dimensional images on consumer, industrial, andmobile electronics such as video projectors, televisions, monitors,gaming systems, or a personal digital assistant (PDA).

Three-dimensional display based services allow users to create,transfer, store, and/or consume information in order for users tocreate, transfer, store, and consume in the “real world”. One such useof three-dimensional display based services is to efficiently presentthree-dimensional images on a display.

Three-dimensional display systems have been incorporated in projectors,televisions, notebooks, handheld devices, and other portable products.Today, these systems aid users by displaying available relevantinformation, such as diagrams, maps, or videos. The display ofthree-dimensional images provides invaluable relevant information.

However, displaying information in three-dimensional form has become aparamount concern for the consumer. Displaying a three-dimensional imagethat does not correlate with the real world decreases the benefit ofusing the three-dimensional display systems.

Thus, a need still remains for a three-dimensional display system withimage conversion mechanism to display three-dimensional images. In viewof the ever-increasing commercial competitive pressures, along withgrowing consumer expectations and the diminishing opportunities formeaningful product differentiation in the marketplace, it isincreasingly critical that answers be found to these problems.Additionally, the need to reduce costs, improve efficiencies andperformance, and meet competitive pressures adds an even greater urgencyto the critical necessity for finding answers to these problems.

Solutions to these problems have been long sought but prior developmentshave not taught or suggested any solutions and, thus, solutions to theseproblems have long eluded those skilled in the art.

DISCLOSURE OF THE INVENTION

The present invention provides a method of operation of a displaysystem, including: calculating a focus measure for an original image;calculating a segment mean based on the focus measure for a segment;generating an ordered segment based on the segment mean; generating asegment depth based on the ordered segment; and generating athree-dimensional image with the segment depth for displaying on adevice.

The present invention provides a display system, including: a focuscalculation module for calculating a focus measure for an originalimage; a mean calculation module, coupled to the focus calculationmodule, for calculating a segment mean based on the focus measure for asegment; a segment order module, coupled to the mean calculation module,for generating an ordered segment based on the segment mean; a depthassignment module, coupled to the segment order module, for generating asegment depth based on the ordered segment; and a three-dimensionalgeneration module, coupled to the depth assignment module, forgenerating a three-dimensional image with the segment depth fordisplaying on a device.

Certain embodiments of the invention have other steps or elements inaddition to or in place of those mentioned above. The steps or elementswill become apparent to those skilled in the art from a reading of thefollowing detailed description when taken with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a display system with image conversion mechanism in anembodiment of the present invention.

FIG. 2 is an exemplary block diagram of the device.

FIG. 3 is an example of an operation of the display system.

FIG. 4 is a control flow of the display system.

FIG. 5 is an example of the three-dimensional image.

FIG. 6 is a flow chart of a method of operation of the display system ina further embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

The following embodiments are described in sufficient detail to enablethose skilled in the art to make and use the invention. It is to beunderstood that other embodiments would be evident based on the presentdisclosure, and that system, process, or mechanical changes may be madewithout departing from the scope of the present invention.

In the following description, numerous specific details are given toprovide a thorough understanding of the invention. However, it will beapparent that the invention may be practiced without these specificdetails. In order to avoid obscuring the present invention, somewell-known circuits, system configurations, and process steps are notdisclosed in detail.

The drawings showing embodiments of the system are semi-diagrammatic andnot to scale and, particularly, some of the dimensions are for theclarity of presentation and are shown exaggerated in the drawing FIGS.Similarly, although the views in the drawings for ease of descriptiongenerally show similar orientations, this depiction in the FIGS. isarbitrary for the most part. Generally, the invention can be operated inany orientation.

The term “module” referred to herein include software, hardware, or acombination thereof. For example, the software can be machine code,firmware, embedded code, and application software. Also for example, thehardware can be circuitry, processor, computer, integrated circuit,integrated circuit cores, a camera, a camcorder, amicroelectromechanical system (MEMS), passive devices, or a combinationthereof.

Some approaches utilize segmentation for depth assignment with relativemotion of segments for successive images by image matching. Thus, it canbe considered as a depth-assigning algorithm, which combinessegmentation and motion.

Another approach introduces depth assignment based on disparity spacedistribution (DSD) function. The DSD function is used for a stochasticdescription of disparity of certain segments. Based on an estimated DSD,each segment has same depth value corresponding to a maximum probabilityvalue of DSD. However, the DSD function considers some relationship withneighbors but does not deal with focus information such as edge andfrequency. Embodiments of the present invention provide solutions oranswers to effectively improve generation of three-dimensional imageswithout motion estimation and DSD.

Referring now to FIG. 1, therein is shown a display system 100 withimage conversion mechanism in an embodiment of the present invention.The display system 100 can include a device 104. The device 104 isdefined as an electronic machine capable of storing and computingdigital data. For example, the device 104 can be of any of a variety ofmobile devices, such as a cellular phone, a personal digital assistant,a tablet, a notebook computer, a tablet PC, a tabletop computer, a smartsurface, or other multi-functional mobile communication or entertainmentdevice.

In another example, the device 104 can be an electronic machine, such asa camera, a mainframe, a server, a cluster server, rack mounted server,or a blade server, or as more specific examples, an IBM System z10™Business Class mainframe or a HP ProLiant ML™ server. Yet anotherexample, the device 104 can be a specialized machine, such as astreaming entertainment device, a portable computing device, a digitalcamera, a thin client, a notebook, a netbook, a smartphone, personaldigital assistant, or a cellular phone, and as specific examples, aSamsung Galaxy Tab™, a Samsung 55″ Class LED 8000 Series Smart TV, aSamsung 3D Blu-ray Disc™ Player, an Apple iPad™, an Apple iPhone™, aPalm® Centro™, or a MOTO Q™ global.

The device 104 can be a standalone device, or can be incorporated with alarger electronic system, for example a home theatre system, a personalcomputer, or a vehicle. The device 104 can be coupled to a communicationpath 106 to communicate with external devices, such as an externaldisplay 108 and a capture device 110.

The communication path 106 is defined as an interconnection betweenelectronic terminals. The communication path 106 can be a variety ofnetworks. For example, the communication path 106 can include wirelesscommunication, wired communication, optical, ultrasonic, or thecombination thereof. Satellite communication, cellular communication,Bluetooth, Infrared Data Association standard (IrDA), wireless fidelity(WiFi), and worldwide interoperability for microwave access (WiMAX) areexamples of wireless communication that can be included in thecommunication path 106. Ethernet, digital subscriber line (DSL), fiberto the home (FTTH), and plain old telephone service (POTS) are examplesof wired communication that can be included in the communication path106.

Further, the communication path 106 can traverse a number of networktopologies and distances. For example, the communication path 106 caninclude direct connection, personal area network (PAN), local areanetwork (LAN), metropolitan area network (MAN), wide area network (WAN)or any combination thereof.

The external display 108 is defined as a device for displaying storedimages of the display system 100. The external display 108 can be, forexample, a 3D TV, a pair of goggles, an LCD screen, or a touch screen.The external display 108 can have observable depths of images and motionimages, and capable of displaying three-dimensionally. The capturedevice 110 is defined as a device for recording images for the displaysystem 100. The capture device 110 can be, for example, a digitalcamera, a camcorder, a webcam, or an array of sensors.

For illustrative purposes, the display system 100 is described with thedevice 104 as a mobile computing device, although it is understood thatthe device 104 can be different types of computing devices. For example,the device 104 can also be a non-mobile computing device, such as aserver, a server farm, or a desktop computer.

Referring now to FIG. 2, therein is shown an exemplary block diagram ofthe device 104. The device 104 can include a user interface 202, acontrol unit 204, and a storage unit 206. The user interface 202 caninclude a display interface 208. The control unit 204 can include acontrol interface 210. The storage unit 206 can include a storageinterface 212.

The user interface 202 allows a user to interface and interact with thedevice 104. The user interface 202 can include an input device and anoutput device. Examples of the input device of the user interface 202can include a keypad, a touchpad, soft-keys, a keyboard, a microphone, atouch pad, a camera, a webcam, or a combination thereof to provide dataand communication inputs.

The user interface 202 can include the display interface 208. Examplesof the output device of the user interface 202 can include the displayinterface 208. The display interface 208 can include a display, aprojector, a video screen, a speaker, or a combination thereof. Thedisplay interface 208 can also be a touch screen, such that inputs canbe received from the display interface 208.

The control unit 204 can execute a software 214 to provide theintelligence of the device 104. The control unit 204 can operate theuser interface 202 to display information generated by the device 104.The control unit 204 can also execute the software 214 for the otherfunctions of the device 104, including receiving image information fromthe capture device 110 of FIG. 1. The control unit 204 can furtherexecute the software 214 for adjusting and updating the imageinformation to display on or through the display interface 208.

The control unit 204 can be implemented in a number of differentmanners. For example, the control unit 204 can be a processor, anembedded processor, a microprocessor, a hardware control logic, ahardware finite state machine, a digital signal processor, or acombination thereof.

The control unit 204 can include the control interface 210. The controlinterface 210 can be used for communication between the control unit 204and other modules in the device 104. The control interface 210 can alsobe used for communication that is external to the device 104.

The control interface 210 can receive information from the other modulesor from external sources, or can transmit information to the othermodules or to external destinations. The external sources and theexternal destinations refer to sources and destinations external to thedevice 104.

The control interface 210 can be implemented in different ways and caninclude different implementations, depending on which modules orexternal units are interfacing with the control interface 210. Forexample, the control interface 210 can be implemented with a pressuresensor, an inertial sensor, a microelectromechanical system, opticalcircuitry, waveguides, wireless circuitry, wireline circuitry, or acombination thereof.

The storage unit 206 can store the software 214. The storage unit 206can also store the relevant information, such as advertisements,preferred settings, operating system, previous adjustments and updates,or a combination thereof.

The storage unit 206 can be a volatile memory, a nonvolatile memory, aninternal memory, an external memory, or a combination thereof. Forexample, the storage unit 206 can be a nonvolatile storage such asnon-volatile random access memory, Flash memory, disk storage, or avolatile storage such as static random access memory.

The storage unit 206 can include the storage interface 212. The storageinterface 212 can be used for communication between the control unit 204and other modules in the device 104. The storage interface 212 can alsobe used for communication that is external to the device 104.

The storage interface 212 can receive information from the other modulesor from external sources, or can transmit information to the othermodules or to external destinations. The external sources and theexternal destinations referred to as sources and destinations externalto the device 104.

The storage interface 212 can be implemented differently depending onwhich modules or external units are being interfaced with the storageunit 206. The storage interface 212 can be implemented with technologiesand techniques similar to the implementation of the control interface210.

Referring now to FIG. 3, therein is shown an example of an operation ofthe display system 100. The display system 100 can process an originalimage 302 to generate a segmentation map 304 and an edge map 306. Theoriginal image 302 depicts a landscape 312 and mountains 314 with a sky316 over the landscape 312 and clouds 318 in the sky 316. Generation ofthe segmentation map 304 and the edge map 306 will be further describedin a subsequent section.

The segmentation map 304 is defined as a representation of the originalimage 302 that has been partitioned into multiple portions or sets ofpixels. The edge map 306 is defined as a representation of the originalimage 302 that identifies points in the original image 302 at whichimage brightness changes sharply or has discontinuities.

The display system 100 can generate a depth map 310, which is defined asa representation of the original image 302 with depths assigned to theportions of the original image 302. The depths are defined as distancesfrom a planar surface providing a three-dimensional image. Generation ofthe depth map 310 will be further described in a subsequent section.

The original image 302 can include multiple features, which are definedas objects that are shown in the original image 302. For example, thefeatures can include human faces, printed characters, vehicles,landscapes, or any other objects that are captured when the originalimage 302 was generated. For a specific example, the features in theoriginal image 302 are shown to include the landscape 312, the mountains314, the sky 316, and the clouds 318.

For example, the depths in the depth map 310 are shown to include alandscape depth 320, a mountain depth 322, a sky depth 324, and a clouddepth 326, which correspond to the landscape 312, the mountains 314, thesky 316, and the clouds 318, respectively. Also for example, order ofthe depths can be in an increasing order of the sky depth 324, the clouddepth 326, the mountain depth 322, and the landscape depth 320 with thelandscape depth 320 having the largest value indicating that thelandscape 312 is closest to a viewer and the sky depth 324 having thesmallest value indicating that the sky 316 is farthest from the viewer.

As an example, FIG. 3 depicts an x-axis and a y-axis that are lines in aplane of the original image 302, the segmentation map 304, the edge map306, and the depth map 310. The x-axis and the y-axis represent ahorizontal line and a vertical line, respectively, along a longer sideand a shorter side, respectively, of the original image 302, thesegmentation map 304, the edge map 306, and the depth map 310.

Referring now to FIG. 4, therein is shown a control flow of the displaysystem 100. The display system 100 can include a segmentation module402, which is defined as a module that partitions the original image 302into multiple segments 404. The segments 404 are defined as groups ofpixels of the original image 302. The segments 404 can be generated fromthe original image 302.

For example, let each of the segments 404 be denoted as R. The originalimage 302 can be partitioned by dividing or segmenting the originalimage 302 into N number of the segments 404, denoted as R₁ . . . R_(N).The segmentation module 402 can generate the segmentation map 304 ofFIG. 3 with a number of the segments 404.

The segmentation module 402 can generate the segments 404 in a number ofways. For example, the segmentation module 402 can includehistogram-based methods. For a specific example, the segments 404 can begenerated using a histogram computed from all of pixels in the originalimage 302 with peaks and valleys in the histogram used to locateclusters of the pixels in the original image 302. Color or intensity canbe used for measurement and identification of the peaks and the valleys.

For another example, the segmentation module 402 can include clusteringmethods. For a specific example, the segments 404 can be generated usinga K-means method, which is an iterative technique that is used topartition an image into K clusters, by assigning each pixel in the imageto the cluster that minimizes the distance between the pixel and thecluster center and re-computing the cluster centers by averaging all ofthe pixels in the cluster.

For yet another example, the segmentation module 402 can generate thesegments 404 using a dynamic template generation. A dynamic template isa set of segmentations, which are generated by segmentation logic. Oneof the segmentations in the dynamic template can share similar depthsand thus can share same degrees of focus.

The display system 100 can include a focus estimation module 406, whichis defined as a module that determines image clarity for pixels orpositions on the original image 302. The focus estimation module 406 caninclude a filter module 408, which is defined as a module that measuresmagnitudes of frequency changes of the original image 302.

The filter module 408 can generate a filter response 409, which isdefined as a frequency response having predetermined frequencycomponents of the original image 302. The predetermined frequencycomponents are defined as frequencies that are above a cutoff frequency,which is defined as a frequency at which output voltage amplitude is −3dB of input voltage amplitude. For example, the filter response 409 canrepresent an arbitrary focus measure function of the filter module 408.

A focus measure function can include any methods of measuring a degreeof focus. For example, an out-of-focus area can be blurry, and anin-focus area can be sharp. Thus, the degree of focus can be estimatedby measuring high frequency components in a target area.

The filter module 408 measures magnitudes of frequency changes of pixelcharacteristics at or near image pixels. The pixel characteristics aremeasurable qualities of pixels of the original image 302. For example,the pixel characteristics can include color, intensity, texture, tone,saturation, or a combination thereof. Also for example, the filtermodule 408 can include a high pass filter to measure magnitudes of highfrequency changes of the pixel characteristics.

The filter response 409 can include a representation of the originalimage 302 with texture information. Texture information refers to highfrequency areas or high frequency components of the original image 302that are passed through the filter module 408. The high frequency areasinclude details resulting in a sharpened image.

The display system 100 can include a focus calculation module 410, whichis defined as a module that determines clarity of an image or portionsof the image by calculating focus measures 412 for positions 414 on theoriginal image 302. Clarity is defined as lack of blurriness or a degreeof focus of an object in an image. The focus measures 412 are defined asmagnitudes determined by assigning each image point to a quantifiablemagnitude of clarity. For example, the focus measures 412 are edgestrengths, which are degrees of focus, of the original image 302.

The focus measures 412 can include measures of how closely light raysoriginating from a surface point of an object converge to the imagepoint. The focus calculation module 410 can generate the edge map 306 ofFIG. 3 based on the filter response 409 and the focus measures 412.

The positions 414 are defined as specific pixels within an image. Forexample, the positions 414 can be represented by coordinates along anx-axis and a y-axis, as depicted in FIG. 3, for a two-dimensional (2D)image including the original image 302.

As a specific example, a method of generating the focus measures 412 forthe positions 414, one of which is denoted as (x,y), in the originalimage 302 can be expressed by the following equation.H(x,y)=F(x,y)●I(x,y)  (1)

where I(x,y) is the original image 302, F(x,y) is the filter response409 of the filter module 408, ● denotes a general operation betweenfunctions including convolution, and H(x,y) is a function that describesa degree of focus at the point of (x,y).

The focus measures 412 can be generated with the function H(x,y). Forexample, the focus measures 412 can be generated by calculating aconvolution of the original image 302 and the filter response 409 ateach of the positions 414.

The display system 100 can include a mean estimation module 415, whichis defined as a module that calculates an average of the focus measures412 for each of the segments 404 in the segmentation map 304. Thedisplay system 100 can process averages of the focus measures 412 of thesegments to generate the depth map 310 of FIG. 3. For example, the depthmap 310 can include the landscape depth 320 of FIG. 3, the mountaindepth 322 of FIG. 3, the sky depth 324 of FIG. 3, and the cloud depth326 of FIG. 3.

The mean estimation module 415 can include a sum-of-degree module 416,which is defined as a module that determines a sum of degrees of focusat each position in a segment of the two-dimensional image. Thesum-of-degree module 416 can generate a focus degree sum 418, which isdefined as a sum of degrees of focus at each of the positions 414 in oneof the segments 404. The focus degree sum 418 can be expressed by thefollowing equation.focus_degree_sum=Σ_((x,y)εR) _(k) H(x,y)  (2)

where H(x,y) is one of the focus measures 412, (x,y) is one of thepositions 414, denotes a member of, R_(k) is the k^(th) or one of thesegments 404, and Σ denotes a summation operation.

The focus degree sum 418 of one of the segments 404 can be calculatedbased on the focus measures 412 and the segments 404. The focus degreesum 418 of one of the segments 404 can be calculated by calculating asummation of the focus measures 412 at the positions 414 in the one ofthe segments 404.

The mean estimation module 415 can include a sum-of-pixel module 420,which is defined as a module that determines a sum of pixels in asegment of the two-dimensional image. The sum-of-pixel module 420 cangenerate a segment pixel sum 422, which is defined as a sum of a numberof pixels in one of the segments 404. The segment pixel sum 422 can beexpressed by the following equation.segment_pixel_sum=Σ_((x,y)εR) _(k)   (3)

where (x,y) is one of the positions 414, denotes a member of, R_(k) isthe k^(th) or one of the segments 404, and Σ denotes a summationoperation.

The segment pixel sum 422 can be calculated based on the segments 404.The segment pixel sum 422 can be calculated by calculating a summationof a number of the positions 414 in the one of the segments 404.

The mean estimation module 415 can include a mean calculation module424, which is defined as a module that determines an average of a numberof the focus degree sum 418 of all of the positions 414 in each of thesegments 404. The mean calculation module 424 can generate a segmentmean 426, which is defined as an average of a number of the focus degreesum 418 of all of the positions 414 in each of the segments 404. Thesegment mean 426 can be expressed by the following equation.

$\begin{matrix}{S_{k} = {\frac{1}{{segment\_ pixel}{\_ sum}} \times {focus\_ degree}{\_ sum}}} & (4)\end{matrix}$where S_(k) is the segment mean 426 in the k^(th) region, which is oneof the segments 404. The segment mean 426 can be calculated by dividingthe focus degree sum 418 by the segment pixel sum 422. The segment mean426 can be calculated by substituting the focus degree sum 418 and thesegment pixel sum 422 in equation 4 with equations 2 and 3,respectively, as expressed by the following equation.

$\begin{matrix}{S_{k} = {\frac{1}{\sum\limits_{{({x,y})} \in R_{k}}\;}{\sum\limits_{{({x,y})} \in R_{k}}\;{H\left( {x,y} \right)}}}} & (5)\end{matrix}$

where H(x,y) is one of the focus measures 412, (x,y) is one of thepositions 414, denotes a member of, R_(k) is the k^(th) or one of thesegments 404, and Σ denotes a summation of.

The segment mean 426 can represent a mean of a high frequency componentor a focus measure component for each segmented area. For example, thesegment mean 426 can represent a mean of focus measure in the k^(th)region, denoted as S_(k).

The display system 100 can include a segment order module 428, which isdefined as a module that determines an order of means of focus measurevalues of segmented areas. The segment order module 428 can generateordered segments 430, which are defined as an arrangement or an order ofmeans of focus measure values of segmented areas. The ordered segments430 can be generated by arranging all of the segments 404 in a segmentorder 432 based on a value of the segment mean 426 of each of thesegments 404.

The segment order 432 is defined as an arrangement of segmented areas ofthe two-dimensional image. The segment order 432 can be predetermined byconfiguring the segment order 432 to a known or fixed state prior togenerating the ordered segments 430. The segment order 432 can be storedin the storage unit 206 of FIG. 2 and read by the segment order module428.

The segment order 432 can preferably include an increasing order offocus measure values, starting with the lowest focus measure value andending with the highest focus measure value, although the segment order432 can include any order. For example, the segment order 432 caninclude a decreasing order.

The display system 100 can include a depth assignment module 434, whichis defined as a module that determines a depth value of each segmentedarea. The depth assignment module 434 can generate segment depths 436,which are defined as values of depth of segmented areas. The segmentdepths 436 can be generated based on the ordered segments 430. The depthassignment module 434 can generate the depth map 310 with the segmentdepths 436.

One of the useful cues for depth estimation from a 2D image can includefocus information, such as the focus measures 412. In the 2D image, anin-focus area can include a high depth value, and an out-of-focus areacan include a low depth value.

Depth assignment can be implemented by ordering each of segment patches,such as the segments 404, to generate the ordered segments 430 based onmean values of high frequency components of each of the segment patches.The ordered segments 430 can be generated based on the segment mean 426of a high frequency component of each of the segments 404.

The segment depths 436 can be generated based on the ordered segments430. Thus, the segment depths 436 can be generated based on the segmentorder 432 by which the segment mean 426 of each of the segments 404 isarranged. The higher a value of the segment mean 426, the higher a valueof one of the segment depths 436. For example, one of the segments 404having one of the segment depths 436 with the highest value can beclosest to the viewer.

Depth assignment method is not constrained to any particular method.Depth assignment method can include any method that assigns a value toeach of the segment depths 436 for each of the ordered segments 430. Fora specific example, one of the segment depths 436 at pixel (x,y), whichis one of the positions 414, can be expressed by the following equation.

$\begin{matrix}{\left. {D\left( {x,y} \right)} \right|_{{({x,y})} \in R_{k}} = {\frac{A}{S_{\max}}S_{k}}} & (6)\end{matrix}$

where A is an arbitrary or predetermined gain, S_(max) is the maximumvalue of S_(k) for k={1, . . . , N} with N denotes a total number of thesegments 404, S_(k) is the segment mean 426 of the k^(th) of thesegments 404, (x,y) is one of the positions 414, denotes a member of,R_(k) is the k^(th) or one of the segments 404, and D(x,y) is one of thesegment depths 436 at one of the positions 414.

Let S_((i)) denote the i^(th) smallest focus measure value among {S₁,S₂, . . . , S_(N)}, where S₍₁₎<S₍₂₎< . . . <S_((N)). The segment depths436 can be assigned to each of the ordered segments 430, denoted asS_((i)). The segment depths 436 can be expressed by the followingequation.D ₁ <D ₂ < . . . <D _(N)  (7)

where {D₁, D₂, . . . , D_(N)} represents a predetermined set of depthvalues for {S₁, S₂, . . . , S_(N)}. The predetermined set of depthvalues are configured to known or fixed values prior to assigning thesegment depths 436 to each of the ordered segments 430, denoted asS_((i)). Assignment of the segment depths 436 provides a novel way toassign depth to two-dimensional images.

For example, let the features in the original image 302 depicted in FIG.3 include an order of the sky 316 of FIG. 3, the clouds 318 of FIG. 3,the mountains 314 of FIG. 3, and the landscape 312 of FIG. 3 with thelandscape 312 represents an object that is in focus more than otherfeatures. The landscape 312 also represents the object that is closestto a viewer than the other features. The depth assignment module 434 cangenerate the segment depths 436, one of which has the largest value andis assigned to the landscape depth 320 of FIG. 3 and another of whichhas the smallest value and is assigned to the sky depth 324 of FIG. 3.

The display system 100 can include a three-dimensional generation module438, which is defined as a module that determines a three-dimensionalimage 440. The three-dimensional image 440 is defined as an imagegenerated with information from a two-dimensional image and depthvalues. As described above, the display system 100 can include a methodfor ordering the segment depths 436 of segmented objects, such as theordered segments 430, with a scene in the original image 302 havingtexture information to generate the three-dimensional image 440.

The three-dimensional generation module 438 can generate thethree-dimensional image 440 with the original image 302 and the segmentdepths 436. The three-dimensional image 440 can be processed and storedin the storage unit 206 for displaying on the device 104 of FIG. 1, theexternal display 108 of FIG. 1, or a combination thereof.

It is understood that the display system 100 can include any orderingmethod. It is also understood that the display system 100 can includeany depth assigning method.

The control flow can optionally include a check for a number ofconditions at the beginning of the control flow. For example, thecontrol flow can include a check for the original image 302 having ascene full of texture or full of high frequency areas. Also for example,the control flow can include a check for the original image 302 having ascene with an out-of-focus object closer to a viewer than an in-focusobject. When any of the checks mentioned above is detected, the displaysystem 100 can optionally bypass the modules described above andgenerate the three-dimensional image 440 to the same as the originalimage 302 without any of the segment depths 436 generated.

It has been discovered that the segment order module 428 generating theordered segments 430 with the segment order 432 for the depth assignmentmodule 434 to assign the segment depths 436 provides improved 2D-to-3Dconversion.

It has also been discovered that distinctive depth information, such asthe segment depths 436, can be effectively assigned to the orderedsegments 430 by utilizing segmentation information, such as the segments404 extracted from the original image 302 and generated with thesegmentation module 402, thereby eliminating problems caused by focusmeasurement not distinctive from object to object in cases where it isvery crucial to have depth perception.

It has further been discovered that utilizing good measure focus, suchas the edge map 306 and the focus measures 412 generated by the focuscalculation module 410, provides an effective method of generating andassigning depths, such as the segment depths 436, to generate thethree-dimensional image 440 with the original image 302 and the segmentdepths 436 for 2D-to-3D conversion.

It has yet further been discovered that a major advantage is providedwith dynamic template generation using segmentation algorithm togenerate the segments 404 with the segmentation module 402 and the focusmeasures 412 with the focus calculation module 410 without any basicdepth model previously given thereby providing effective generation ofthe segment depths 436 with the segment mean 426 for any arbitraryimage, such as the original image 302.

It has yet further been discovered that the focus degree sum 418generated by the sum-of-degree module 416 and the segment pixel sum 422generated by the sum-of-pixel module 420 provide generation of thesegment mean 426 to effectively calculate the ordered segments 430.

It has yet further been discovered that the filter module 408 allowsaccurate calculation of the focus measures 412 by the focus calculationmodule 410.

It has yet further been discovered that generating the depth map 310with the segment depths 436 based on the ordered segments 430 providesthe display system 100 with a smoother depth estimation therebyeliminating three-dimensional visual defects.

The focus calculation module 410 can be coupled to the filter module408. The focus calculation module 410 and the segmentation module 402can be coupled to the sum-of-degree module 416.

The sum-of-pixel module 420 can be coupled to the segmentation module402. The sum-of-pixel module 420 and the sum-of-degree module 416 can becoupled to the mean calculation module 424.

The segment order module 428 can be coupled to the mean calculationmodule 424 and the depth assignment module 434. The depth assignmentmodule 434 can be coupled to the three-dimensional generation module438.

The segmentation module 402 can be implemented with the user interface202 of FIG. 2, the control unit 204 of FIG. 2, the control interface 210of FIG. 2, the storage unit 206, the storage interface 212 of FIG. 2,and the software 214 of FIG. 2. For example, the user interface 202, thecontrol interface 210, or a combination thereof can be implemented toreceive the original image 302. Also for example, the control unit 204,the storage interface 212, the software 214, or a combination thereofcan be implemented to generate the segmentation map 304 and the segments404.

The filter module 408 can be implemented with the user interface 202,the control unit 204, the control interface 210, the storage unit 206,the storage interface 212, and the software 214. For example, the userinterface 202, the control interface 210, or a combination thereof canbe implemented to receive the original image 302. Also for example, thecontrol unit 204, the storage interface 212, the software 214, or acombination thereof can be implemented to measure magnitudes offrequency changes of the pixel characteristics of the original image302.

The focus calculation module 410 can be implemented with the userinterface 202, the control unit 204, the control interface 210, thestorage unit 206, the storage interface 212, and the software 214. Forexample, the user interface 202, the control interface 210, or acombination thereof can be implemented to receive the original image302. Also for example, the control unit 204, the storage interface 212,the software 214, or a combination thereof can be implemented togenerate the edge map 306 and the focus measures 412.

The sum-of-degree module 416 can be implemented with the control unit204, the storage unit 206, the storage interface 212, and the software214. For example, the control unit 204, the storage interface 212, thesoftware 214, or a combination thereof can be implemented to generatethe focus degree sum 418.

The sum-of-pixel module 420 can be implemented with the control unit204, the storage unit 206, the storage interface 212, and the software214. For example, the control unit 204, the storage interface 212, thesoftware 214, or a combination thereof can be implemented to generatethe segment pixel sum 422.

The mean calculation module 424 can be implemented with the control unit204, the storage unit 206, the storage interface 212, and the software214. For example, the control unit 204, the storage interface 212, thesoftware 214, or a combination thereof can be implemented to generatethe segment mean 426.

The segment order module 428 can be implemented with the user interface202, the control unit 204, the control interface 210, the storage unit206, the storage interface 212, and the software 214. For example, theuser interface 202, the control interface 210, or a combination thereofcan be implemented to preset or pre-configure the segment order 432.Also for example, the control unit 204, the storage interface 212, thesoftware 214, or a combination thereof can be implemented to generatethe ordered segments 430.

The depth assignment module 434 can be implemented with the control unit204, the storage unit 206, the storage interface 212, and the software214. For example, the control unit 204, the storage interface 212, thesoftware 214, or a combination thereof can be implemented to generatethe segment depths 436.

The three-dimensional generation module 438 can be implemented with theuser interface 202, the control unit 204, the display interface 208 ofFIG. 2, the control interface 210, the storage unit 206, the storageinterface 212, and the software 214. For example, the user interface202, the display interface 208, the control interface 210, or acombination thereof can be implemented to display the three-dimensionalimage 440. Also for example, the control unit 204, the storage interface212, the software 214, or a combination thereof can be implemented togenerate the three-dimensional image 440.

The physical transformation from displaying the three-dimensional image440 results in movement in the physical world, such as people moving inresponse to the three-dimensional image 440 when playing games orviewing the three-dimensional image 440. The display interface 208 candisplay the three-dimensional image 440 by manipulating pixels at one ofthe positions 414 on the device 104, thus resulting in movement in thephysical world.

The display system 100 describes the module functions or order as anexample. The modules can be partitioned differently. For example, thesegment order module 428 and the depth assignment module 434 can beimplemented together in one module. Each of the modules can operateindividually and independently of the other modules. Furthermore, datagenerated in one module can be used by another module without beingdirectly coupled to each other.

The “modules” described above can be implemented in a number ofdifferent fashions but refer to hardware implementation depending oncontext as used in this application, including the claims which followlater. In apparatus or system claims, the “modules” are hardwareimplementations as specialized hardware blocks separate from those shownin FIG. 2 and can be part of those shown in FIG. 2, such as the controlunit 204 or the display interface 208.

Referring now to FIG. 5, therein is shown an example of thethree-dimensional image 440. The three-dimensional image 440 is depictedwith the features in the original image 302 of FIG. 3 including thelandscape 312, the mountains 314, the sky 316, and the clouds 318.

The landscape 312, the mountains 314, the sky 316, and the clouds 318are shown in the three-dimensional image 440 based on the landscapedepth 320 of FIG. 3, the mountain depth 322 of FIG. 3, the sky depth 324of FIG. 3, and the cloud depth 326 of FIG. 3, respectively. Forillustration purposes, depths of the features are represented by adensity of horizontal lines in the three-dimensional image 440 as anexample.

The density of the horizontal lines is a number of the horizontal linesper a unit area in the three-dimensional image 440. For example, afeature having the lowest depth value can be considered farthest fromthe viewer, and thus, its depth can be represented by the highestdensity of the horizontal lines. Also for example, a feature having thehighest depth value can be considered closest to the viewer, and thus,its depth can be represented by the lowest density of the horizontallines.

As an example, an order of the depths can be in an increasing order ofthe sky depth 324, the cloud depth 326, the mountain depth 322, and thelandscape depth 320. As a specific example, the sky 316 is shown havingthe highest density of the horizontal lines for being farthest from theviewer, and the landscape 312 is shown having the lowest density of thehorizontal lines for being closest to the viewer.

Referring now to FIG. 6, therein is shown a flow chart of a method 600of operation of the display system 100 in a further embodiment of thepresent invention. The method 600 includes: calculating a focus measurefor an original image in a block 602; calculating a segment mean basedon the focus measure for a segment in a block 604; generating an orderedsegment based on the segment mean in a block 606; generating a segmentdepth based on the ordered segment in a block 608; and generating athree-dimensional image with the segment depth for displaying on adevice in a block 610.

Thus, it has been discovered that the display system of the presentinvention furnishes important and heretofore unknown and unavailablesolutions, capabilities, and functional aspects for a display systemwith image conversion mechanism. The resulting method, process,apparatus, device, product, and/or system is straightforward,cost-effective, uncomplicated, highly versatile, accurate, sensitive,and effective, and can be implemented by adapting known components forready, efficient, and economical manufacturing, application, andutilization.

Another important aspect of the present invention is that it valuablysupports and services the historical trend of reducing costs,simplifying systems, and increasing performance.

These and other valuable aspects of the present invention consequentlyfurther the state of the technology to at least the next level.

While the invention has been described in conjunction with a specificbest mode, it is to be understood that many alternatives, modifications,and variations will be apparent to those skilled in the art in light ofthe aforegoing description. Accordingly, it is intended to embrace allsuch alternatives, modifications, and variations that fall within thescope of the included claims. All matters hithertofore set forth hereinor shown in the accompanying drawings are to be interpreted in anillustrative and non-limiting sense.

What is claimed is:
 1. A method of operation of a display systemcomprising: calculating a focus measure for an original image;calculating a segment mean based on the focus measure for a segment ofthe original image; generating an ordered segment, with a control unit,based on the segment mean; generating a segment depth based on theordered segment, given by:D(x,y)|_((x,y)εR) _(k) =A/S _(max) S _(k) wherein D(x,y) is the segmentdepth, A is a gain, S_(max) is a maximum value of the segment mean forthe original image, and S_(k) is the segment mean for the segment; andgenerating a three-dimensional image with the segment depth fordisplaying on a device.
 2. The method as claimed in claim 1 whereingenerating the ordered segment includes generating the ordered segmentby arranging the segment in a segment order.
 3. The method as claimed inclaim 1 further comprising calculating a focus degree sum based on thefocus measure and the segment.
 4. The method as claimed in claim 1further comprising calculating a segment pixel sum based on the segment.5. The method as claimed in claim 1 further comprising generating afilter response of the original image.
 6. A method of operation of adisplay system comprising: calculating a focus measure for an originalimage; calculating a segment mean based on the focus measure for asegment of the original image; generating an segment, with a controlunit, based on the segment mean; generating a segment depth based on theordered segment, given by:D(x,y)|_((x,y)εR) _(k) =A/S _(max) S _(k) wherein D(x,y) is the segmentdepth, A is a gain, S_(max) is a maximum value of the segment mean forthe original image, and S_(k) is the segment mean for the segment; andgenerating a three-dimensional image with the original image and thesegment depth for displaying on a device.
 7. The method as claimed inclaim 6 wherein generating the ordered segment includes generating theordered segment by arranging the segment in a segment order with thesegment order predetermined.
 8. The method as claimed in claim 6 furthercomprising calculating a focus degree sum by calculating a summation ofa number of the focus measure in the segment.
 9. The method as claimedin claim 6 further comprising calculating a segment pixel sum bycalculating a summation of positions in the segment.
 10. The method asclaimed in claim 6 further comprising: generating a filter response ofthe original image; and wherein: calculating the focus measure includescalculating a convolution of the original image and the filter response.11. A display system comprising: a user interface configured to receivean original image; and a control unit, coupled to the user interface,configured to: generate a focus measure for the original image; generatea segment mean based on the focus measure for a segment of the originalimage; generate an ordered segment based on the segment mean; generate asegment depth based on the ordered segment, given by:D(x,y)|_((x,y)εR) _(k) =A/S _(max) S _(k) wherein D(x,y) is the segmentdepth, A is a gain, S_(max) is a maximum value of the segment mean forthe original image, and S_(k) is the segment mean for the segment; andgenerate a three-dimensional image with the segment depth for displayingon a device.
 12. The system as claimed in claim 11 wherein the controlunit is configured to generate the ordered segment by arranging thesegment in a segment order.
 13. The system as claimed in claim 11wherein the control unit is configured to calculate a focus degree sumbased on the focus measure and the segment.
 14. The system as claimed inclaim 11 wherein the control unit is configured to calculate a segmentpixel sum based on the segment.
 15. The system as claimed in claim 11wherein the control unit is configured to generate a filter response ofthe original image.
 16. The system as claimed in claim 11 wherein thecontrol unit is configured to generate the three-dimensional image withthe original image and the segment depth for displaying on the device.17. The system as claimed in claim 16 wherein the control unit isconfigured to generate the ordered segment by arranging the segment in asegment order with the segment order predetermined.
 18. The system asclaimed in claim 16 wherein the control unit is configured to calculatea focus degree sum by calculating a summation of a number of the focusmeasure in the segment.
 19. The system as claimed in claim 16 whereincontrol unit is configured to calculate a segment pixel sum bycalculating a summation of positions in the segment.
 20. The system asclaimed in claim 16 wherein the control unit is configured to: generatea filter response of the original image; and calculate a convolution ofthe original image and the filter response.